Extending Contrastive Learning to Unsupervised Coreset Selection

Self-supervised contrastive learning offers a means of informative features from pool unlabeled data. In this paper, we investigate another useful approach. We propose an entirely coreset selection method. regard, learning, one several self-supervised methods, was recently proposed and has consistently delivered the highest performance. This prompted us to choose two leading methods for learning: simple framework visual representations (SimCLR) momentum (MoCo) framework. calculated cosine similarities each example epoch entire duration process subsequently accumulated similarity values obtain score. Our assumption that sample with low would likely behave as coreset. Compared existing labels, our approach reduced cost associated human annotation. study, unsupervised method implemented achieved improvements 1.25% (for CIFAR10), 0.82% SVHN), 0.19% QMNIST) over randomly selected subset size 30%. Furthermore, results are comparable those supervised methods. The differences between above mentioned (forgetting events) were 0.81% on CIFAR10 dataset, −2.08% SVHN dataset (the outperformed method), 0.01% QMNIST at addition, exhibited robustness even if model target not identical (e.g., using ResNet18 ResNet101 model). Lastly, obtained more concrete proof examples highly by showing performance gap non-coreset samples in cross test experiment. observed pair ((testing: non-coreset, training: coreset), (testing: coreset, non-coreset)), i.e. (94.27%, 67.39 %) CIFAR10, (98.24%, 83.30%) SVHN, (99.89%, 93.07%)